Some Peculiar Semi-markov Processes

The KTH Visit in Semi-Markov Processes

Overshoot Functionals for Almost Semi-continuous Processes Defined on a Markov Chain

The distribution of extrema and overjump functionals for the semi-continuous processes (processes that intersect positive or negative level continuously) on a Markov chain were considered by many authors( for instance, see [1] [3]). In the paper [4] the distribution of extrema for almost semi-continuous processes were treated (the processes that intersect positive or negative level by exponenti...

Asymptotic Behavior of Multivariate Reward Processes with Nonlinear Reward Functions

Expected Duration of Dynamic Markov PERT Networks

Abstract : In this paper , we apply the stochastic dynamic programming to approximate the mean project completion time in dynamic Markov PERT networks. It is assumed that the activity durations are independent random variables with exponential distributions, but some social and economical problems influence the mean of activity durations. It is also assumed that the social problems evolve in ac...

Target-sensitive control of Markov and semi-Markov processes

We develop the theory for Markov and semi-Markov control using dynamic programming and reinforcement learning in which a form of semi-variance which computes the variability of rewards below a pre-specified target is penalized. The objective is to optimize a function of the rewards and risk where risk is penalized. Penalizing variance, which is popular in the literature, has some drawbacks that...